Methods and systems to provide a playlist for simultaneous presentation of a plurality of media assets

ABSTRACT

Systems and methods are described herein for generating a playlist for a simultaneous presentation of a plurality of media assets. The system retrieves a user preference associated with a user profile and receives a selection of a first media asset and a second media asset from the plurality of media assets for presentation on a user device. The system parses the respective audio streams of the first media asset and the second media asset to identify one or more preferred audio segments based on the user preference and generates the playlist of the identified one or more preferred audio segments. Based on a generated audio playlist, the system generates, for presentation on the user device, the video stream for each of the first media asset and the second media asset and the playlist of the identified one or more preferred audio segments.

BACKGROUND

The present disclosure relates to methods and systems for consumingmultiple media assets and, more particularly, to methods and systems forgenerating a combined audio stream based on user preferences from themultiple media assets for presentation on a consumer device. In anotherexample, the present disclosure is directed to methods and systems foridentifying events within a media asset and proposing to present theevent on display.

SUMMARY

Users often enjoy experiencing multiple media assets, such as watchingmultiple football games on television; however it requires switchingback and forth between channels. This becomes more of a challenge withthe wide availability and selection of digital content, combined with alarge number of electronic devices available for its playback, meaningthat consumers may select and play multiple media assets on severalscreens of a display or on multiple displays. This allows users greatflexibility in where and how they may view their content. Some users maystream multiple media assets on a single device with multiple screenssimultaneously. In some scenarios, viewers may wish to view a firstmedia asset and listen to a second media asset. In other scenarios,viewers may wish to view two different media assets and listen to athird media asset. In yet other scenarios, viewers may wish to viewcertain media assets while listening to other media assets. Often,viewers need to select a media asset to listen to and media assets toview. In a scenario where the viewer prefers to listen to parts of bothmedia assets, the viewer needs to manually switch back and forth betweenthe media assets, which may be laborious and inconvenient. In addition,when multiple media assets are available for display, it may bedifficult to determine which audio from each media asset to present forthe viewers. Providing a display with multiple media assets for viewerswithout requiring a viewer selection may improve a viewer’s ability toaccess media, particularly for media that is important or interestingfor the viewer.

In one scenario, “Multi-view” is a feature that is commonly used by somevideo providers to enable a user to watch multiple content items, e.g.,sport games, simultaneously. For example, if “Dual-view” is enabled,then the user can watch two games side by side. Similarly, if “Thumbnailview” is enabled, then the user can watch multiple games, where a largevideo for one game is displayed, and the remaining games are shown onthe side as small videos, or even a quad view - where all four videosare the same size. Watching multiple games at once can be inconvenientfor some users and even distracting since a user can miss, for example,key moments or interesting plays in a game. Similarly, the client devicemight experience poor bandwidth connection, resulting in bad-qualityvideo and causing frames to be dropped, particularly when otherapplications are running in the background.

The advancement of digital transmission of media content has increasedthe amount of data that can be transmitted. In particular, media systemsmay transmit metadata that includes detailed information about mediacontent. This metadata may include descriptions and tags to eventswithin the media asset. While this information, by itself, is notuseable for displaying to a human user, computer processors can read andinterpret this information. However, while computer processors may readand interpret this information and generate media content for displaythat may be useable by a human user (e.g., output media content on adisplay screen in a format recognizable to humans), these systems stillfail to solve the problem of customizing which media asset to present tothe viewer at a given moment. That is, as multiple media assets areidentified as desired by the viewer, current systems lack the ability todetect specific triggers for when to switch between the various mediaassets to display for presentation based on the particular preference ofthe viewer.

In all of these scenarios, however, the ability to consume multipledifferent content streams concurrently is not without its difficulties.In particular, devices currently leave it to users to resolve their ownconflicts, which can lead to excessive work and frustration. Forinstance, when users discover other content while listening to apodcast, they often want to pause the podcast while viewing the newcontent. In another example, when users consume multiple sporting events(e.g., a football game and a basketball game) simultaneously, the systemmay present the video of each sporting event but is limited in itsability to switch the audio between the sporting events. That is, userscan switch back and forth in sporting events by simply switching whichscreen they are looking at. However, users currently must manuallyswitch between the programs to listen to the audio from both programs.The need to manually implement media asset settings when multiplecontent streams are playing is, as above, a source of user frustration.

Accordingly, to overcome the problems and limitations of such servicesand their interfaces, systems and methods are described herein for acomputer-based process that automatically determines when an individualis playing or requesting content playback on two different media assets,determines whether the two content streams present an audio conflict,and if so, resolves the conflict. Conflict resolution may be performedby, for instance, switching audio between the media assets or playingaudio of one media asset and muting one of the media asset streams. Ifso, the conflict is automatically resolved. For example, if the systemdetermines that a user is listening to a first media asset, the systemrecognizes that a conflict may exist with the audio of the second mediaasset and causes one of the audio streams to stop. However, if thesystem determines that the same user instead initiates a Ted Talk® videoon a laptop, a conflict is deemed to occur, as people typically cannotpay full attention to each of two simultaneous audio tracks containingspeech. Accordingly, the second media asset is automatically muted whileits video continues. Other examples are further explained below.

In more detail, a list of users is maintained, and the system determineswhen a user on the list is playing a first media stream on a user deviceand requests a second media asset stream on the user device. When a userdoes so, the system then determines whether the audio segments (e.g.,audio tracks) of the two content streams conflict. The system thendetermines which audio segment (e.g., audio tracks) to present duringthe presentation of the two media assets. In some embodiments, the audiomay continuously switch back and forth between the two media assets. Inyet another embodiment, the audio may be from a third media assetdifferent from the two media assets being presented on a display of theuser device. If no conflict is deemed to be present, both contentstreams are transmitted to the user device. For example, one of theaudio streams may have minimal dialogue that does not interfere with thesecond audio stream. In another example, commercials line up in such away that the system may switch back and forth between the two mediaassets between commercials. Conversely, if a conflict is deemed tooccur, this conflict is resolved by muting or redirecting one of theaudio content streams. Which audio content stream to mute or redirect isbased on the user preferences (e.g., team, show, player or actor, etc.).

In some embodiments, methods and systems are disclosed for generating aplaylist from a simultaneous presentation of a plurality of mediaassets. The system includes control circuitry configured to retrieve auser preference associated with a user profile. The system furtherincludes input and output circuitry configured to receive a selection ofa first media asset and a second media asset from the plurality of mediaassets for presentation on a user device. Each of the first media assetand the second media asset includes a video stream and an audio stream.The control circuitry then parses the respective audio streams of thefirst media asset and the second media asset to identify one or morepreferred audio segments based on the user preference. The system thengenerates the playlist of the identified one or more preferred audiosegments. Based on the generated playlist, the system then generates,for presentation on the user device, the video stream for each of thefirst media asset and the second media asset and the playlist of theidentified one or more preferred audio segments. In some embodiments,the system generates the media assets for presentation simultaneously.

In some embodiments, the control circuitry generates the playlist of theidentified one or more preferred audio segments by combining one or morepreferred audio segments from the first media asset with one or morepreferred audio segments from the second media asset. In someembodiments, the control circuitry generates the playlist of theidentified one or more preferred audio segments by interleaving one ormore first preferred audio segments and one or more second preferredaudio segments to create a combined preferred audio stream. In someembodiments, the interleaving one or more first preferred audio segmentsand one or more second preferred audio segments to create the combinedpreferred audio stream includes arranging each of the one or morepreferred audio segments of the first media asset and the second mediaasset in a sequence to create a storyline matching the user preference.

In some embodiments, the methods and systems disclosed propose enablinga parallel view of two channels either side by side or aspicture-in-picture (PIP) based on the user’s interest /preference forthe mode and have the merged/combined audio for these two channelsrendered.

In some embodiments, methods and systems are disclosed where the system,via a control circuitry, retrieves a user preference associated with auser profile. The system identifies, based on user preference, aplurality of media assets, for example, movies or shows the user prefersto watch based on viewer history. For display on the user device, thesystem then generates a first media asset of the plurality of mediaassets. For example, a first show is presented on the display. Thesystem accesses metadata of a second media asset, to identify a segmentof the second media asset including an event. The event may beidentified based on the user’s preference or metadata indicative of theimportance of the event—for example, overtime of a favorite sport. Theevent may be based on the characters in the show. Based on the metadata,the system determines a playback time of the segment of the second mediaasset including the event. During the playback time of the segment ofthe second media asset including the event, the system generates fordisplay on the user device the segment of the second media asset.

In some embodiments, the system performs an analysis of the second mediaasset. The analysis is selected from one or more of an object analysis,a frame analysis, a subtitle analysis, a speech analysis, and a naturallanguage processing NLP analysis. The system compares each segment ofthe second media asset with user preferences. The system determines,based on the comparing, that a segment of the second media asset matchesthe user preferences. In some embodiments, the system generates fordisplay, on the user device in a picture-and-picture mode, the segmentof the second media asset.

In some embodiments, upon completion of the segment of the second mediaasset, the system automatically switches back to generating for displaythe first media asset.

In some embodiments, methods and systems are disclosed where the systemmay receive a request for a first media asset and a second media assetfor presentation on a user device. Based on receiving the request, thesystem generates a user interface including a first screen configuredfor the presentation of the first media asset and a second screenconfigured for the presentation of the second media asset. For example,the display may include a split screen with one side playing a firstmovie and a second side playing a second movie. The system thenprocesses an audio manifest for the first media asset and the secondmedia asset. Each audio manifest includes a plurality of audio segments,each audio segment being associated with a priority rating. For example,the priority rating measures the importance of a segment or scene, wherea segment in which a team scores or almost scores includes a higherpriority rating as compared to other parts of a game. The system thencompares the priority rating for each audio segment from the first mediaasset with the priority rating for each audio segment from the secondmedia asset. Based on the comparing, the system generates a combinedaudio manifest with audio segments from the first media asset and thesecond media asset based on the audio segment having a higher priorityrating. Subsequent to generating the combined audio, the systemtransmits for presentation on the user device a video stream of thefirst media asset on the first screen, a video of the second media asseton the second screen and the combined audio manifest.

In some embodiments, the system identifies a closed caption for eachaudio segment. The system then continues transmitting a closed captionfor each audio segment determined to have a lower priority rating forpresentation on the user device. In some embodiments, the systemgenerates a combined audio manifest in a user enable mode. For example,the user enables the mode for dual or multi content consuming, and amanifest includes both the audio and the closed caption of both mediaassets

It should be noted that the systems, methods, apparatuses, and/oraspects described above may be applied to or used in accordance withother systems, methods, apparatuses, and/or aspects described in thisdisclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects and advantages of the disclosure will beapparent upon consideration of the following detailed description, takenin conjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIG. 1 shows an exemplary scenario 100 in which multiple audio soundsare identified from a plurality of media assets to be combined forpresentation on a user device, according to some embodiments of thedisclosure;

FIGS. 2A and 2B show an exemplary scenario 200 in which multiple audiosegments are combined from a plurality of media assets, in accordancewith some embodiments of the disclosure;

FIG. 3 shows an alternate exemplary scenario 300 in which multiple audiosegments are identified from a plurality of media assets to be combinedfor presentation on a user device, in accordance with some embodimentsof the disclosure;

FIG. 4 shows an alternate exemplary scenario 400 in which multiple audiomanifests are combined into a single combined audio manifest from aplurality of media assets, in accordance with some embodiments of thedisclosure;

FIG. 5 is a block diagram of an illustrative device for combiningmultiple audio manifests from a plurality of media assets to a singlecombined audio manifest, in accordance with some embodiments of thedisclosure;

FIG. 6 depicts a flowchart of an illustrative process in which multipleaudio segments are combined from a plurality of media assets, inaccordance with some embodiments of the disclosure;

FIG. 7 depicts a flowchart of an illustrative process in which multipleaudio segments are combined from a plurality of media assets, inaccordance with some embodiments of the disclosure;

FIG. 8 depicts a flowchart of an illustrative process in which multipleaudio segments are combined from a plurality of media assets, inaccordance with some embodiments of the disclosure;

FIG. 9 is a block diagram showing components and data flow therebetweenof a system for combining multiple audio sounds from a plurality ofmedia assets, in accordance with some embodiments of the disclosure;

FIG. 10 is a block diagram of an illustrative system for combiningmultiple audio sounds from a plurality of media assets, in accordancewith some embodiments of the disclosure; and

FIG. 11 shows an illustrative diagram of a homogeneous audio combinerexemplary signaling mechanism, in accordance with some embodiments ofthe disclosure.

DETAILED DESCRIPTION

Systems and methods are provided herein to improve and alleviatebandwidth constraints on the many media assets being consumed inmulti-view modes or various other modes where multiple media assets areconsumed simultaneously. In some embodiments, a “multi-view” service(e.g., cloud-based) dynamically creates (e.g., instantiates) one or moreadditional video players and dynamically dismantles at least one videoplayer, based on multiple factors, such as network conditions, userpreferences for content (e.g., sports, teams, etc.), and real-timeevents (e.g., a potential play in a sports game that can result in alead or score change -red zone play, etc.). Video players include amanifest of a media asset including video stream and audio stream andassociated metadata for the media asset. The dynamic multi-viewinterface can play (e.g., toggle through) various portions of differentlive games in a single interface/video player based on an excitementscore associated with a given game at that time. For example, theremight be two different NFL games available for streaming - however, animportant or exciting play may be happening in Game A, while there’s noreal action occurring in other games that the user “subscribes to” toreceive updates on. In such a case, the live video of Game A is playedback. However, if there are multiple potentially exciting playsoccurring in two or more games, then the multi-view interface isdynamically expanded to show a live stream of the other games. Forexample, if two games need to be presented, then the multi-view switchesto a “dual-view” to present the two games side by side. In such ascenario, the system instantiates a second video player and assigns thestream of the second game to the second player. Similarly, if the systemdetermines that three games (Games A, B, and C) need to be displayed,then the multi-view service instantiates two video players and assignsthe streams associated with games B and C to those video players. Thesize of the video players can be fixed according to a predeterminedconfiguration or dynamically changed based on the excitement factor ortype of play. For example, a game where a play in the end zone is takingplace might be emphasized by getting displayed in a player that islarger in size than the other player(s).

FIG. 1 shows an exemplary scenario 100 in which multiple audio soundsare identified from a plurality of media assets being presented on adisplay. In some embodiments, the system identifies audio sounds theuser prefers. In some embodiments, the system identifies unwanted audiosounds for muting. A display 101 illustrates a presentation of themultiple media assets and audio sounds associated with each media asset,in accordance with some embodiments of the disclosure. In scenario 100,a user device 102 receives multiple media assets for presentation. Themultiple media assets may be a movie, a news report, a weather report, asports report, or a sports event. The user device 102 may be a phone, asmartphone, a tablet, a laptop computer, a desktop computer, or anyother device capable of presenting multiple media assets forconsumption, whether live, recorded or streamed over the Internet. Inone example, the user device 102 receives two football games forpresentation where a first media asset 101 (e.g., Jets football game) isa presentation on the left side of the display on user device 102 and asecond media asset 110 (e.g., Green Bay football game) is a presentationon the right on the user device 102. During the presentation of themultiple media assets on the user device 102, the system parses therespective audio streams for each of the multiple media assets toidentify one or more preferred audio segments based on the userpreferences. For example, the system may parse the audio stream 105identified by the time bar of the first media asset 101 into respectiveaudio segments 1MA1, 2MA1, 3MA1, 4MA1 and 5MA1. Although five audiosegments are illustrated, such an example is non-limiting and may be anynumber of segments that are parsed by the system. The audio stream 105may correspond to the audio of the first media asset 101. The system maygenerate different voice profiles for sounds within the audio stream105. The audio stream 106 corresponds to the audio for the second mediaasset 110. For example, the system may parse the audio stream 106identified by the time bar of the first media asset 110 into respectiveaudio segments 1MA2, 2MA2, 3MA2, 4MA2 and 5MA2.

The system performs an analysis on the audio segments in the processingcircuitry 906, which includes an audio module 110 configured to monitoraudio of the first media asset and the second media asset and a userinterface 112 for inputting user preferences. The processing circuitrymay be communicatively connected to server database 114 to identifydifferent segments and a user preference database 108 configured tostore preferences indicated by the user. Upon processing the audiostreams for both the first and second media assets, the processingcircuitry generates a combined audio segment 116. The combined audiosegment 116 includes audio segments (1MA1 and 1MA3) from the first mediaasset and the audio segments (2MA2, 4MA2, 5MA2) from the second mediaasset. Once the combined audio segment 116 is generated, it istransmitted to the user device 102 for presentation along with the videoof the first media asset 101 and second media asset 110. The combinedaudio segment 116 is organized in sequential order and generated forpresentation on the user device.

The system based on the aforementioned configurations improves theefficiency and enjoyment of the user experience because the system onlytransmits one audio file to the user device. In some embodiments, theuser preference can be configured or based on learned preferences forthe user. That is, based on previous user interactions with multiplemedia assets, the system may learn preferences for the user. In an eventthat includes multi topics, the system may seek user feedback toconfigure the user preferences. In some embodiments, the userpreferences may be fed back to the broadcaster. Based on a user’schoices of the channels, the system may generate automated audio mixingusing existing NLP algorithms. In some embodiments, an audio segment ofa plurality of media assets may be identified and time synced to boththe events. In some embodiments, the audio segments include timestampsof start time and end time. During the transition between the differentaudio segments, the system may provide additional media assets,transition sounds or a presented original media asset. The identifiedkey audio elements are mixed by retaining the consistent audioparameters and generating combined multi-event audio. In someembodiments, the system performs signaling to transmit two videosmanifest and one audio manifest. At the end-user, both the manifestfiles preferred by the user will be loaded, the video segments will befetched from the individual playlist and the audio will be discarded,but the combined audio will be used instead. A notice of which channelsare combined is present in the combined audio signal in the playlist.Combined audio can be selected for the most popular channels or can bedriven by user preferences. The system may also reduce bitratetransmitted between the server and the user device because only oneaudio stream is transmitted.

In some embodiments, the system transmits from a server to a user devicethe audio and mixed audio-video stream based on user preference formulti-event viewing. The system may continue to transmit as anaudio-only mixed media asset. Based on the transmitted audio and videomixed content, the system identifies a time frame-based/criticalevent-based timeline for presentation on the user device.

The media asset segments for the first media asset and the second mediaasset may be identified based on user preferences for a particular eventor commentator and stored on the server or a database 114. For example,a one- or two-second sample of the voice of a commentator in the firstmedia asset may have been used to identify and generate a voice profilefor the sound of the first media asset. In some cases, several suchsamples may be identified, and an average of each voice characteristicidentified therein is used to parse the audio segments of the firstmedia asset. Alternatively or additionally, the user device 102 or acloud-based server may be prompted to learn the variations in voicecharacteristics of an audio stream for a first media asset and torecognize voices and recommend audio segments for presentation based onuser preferences. In some embodiments, the audio between the two mediaassets may switch based on commercials. In some embodiments, the system,including the user device 102, may identify audible sounds on thedisplay of a media asset based on the movement of a commentator’s mouthto pinpoint which commentator is speaking. In some embodiments, amanifest file containing the information related to the speakers in themedia asset may be transmitted to the consumption device. In someembodiments, the manifest file may include a closed caption receivedwith the media asset that may identify the speaker before each line oftext. For example, the name “Jim” may appear in the caption to indicatethat Jim is speaking. In some embodiments, at the client-server level,the system may build a voice profile or sound profile of the audiostream corresponding to first media asset 101 based on the audio streamaccompanying the media asset. In some embodiments, the user device 102may receive from the server a voice profile or sound profile of thefirst media asset 101 based on the audio stream accompanying the mediaasset presentation while the first media asset 101 is presented fordisplay. In one example, a user may be consuming a number of horrormovies or shows and may prefer to hear (and watch) a media asset when itcontains jolting and dramatic sounds. In such a case, the systemincreases the volume of that media asset and mutes the volume of thebackground sounds or audio of the second media asset.

In some embodiments, media asset data (via a manifest file) from serverdatabase 114 (e.g., content item source) may be provided to user device102 using a client/server approach. For example, user device 102 maypull media asset data from a server (e.g., server database 114), or aserver may push content item data to user device 102. In someembodiments, a client application residing on user device 102 mayinitiate sessions with a user preference database 108 to obtain manifestfiles including audio segments when needed, e.g., when the manifest fileis out of date or when user device 102 receives a request from the userto receive data.

Media asset and/or manifest files delivered to user device 102 may beover-the-top (OTT) media assets. OTT media asset delivery allowsInternet-enabled user devices, such as user device 102, to receive anymedia asset that is transferred over the Internet, including any mediaasset described above, in addition to media assets received over cableor satellite connections. An OTT media asset is delivered via anInternet connection provided by an Internet service provider (ISP), buta third party distributes the media asset. The ISP may not beresponsible for the viewing abilities, copyrights, or redistribution ofthe media asset and may only transfer IP packets provided by the OTTmedia asset provider. Examples of OTT media asset providers includeYouTube™, Netflix™, and Hulu™, which provide audio and video viamanifest file. YouTube™ is a trademark owned by Google Inc., Netflix™ isa trademark owned by Netflix Inc., and Hulu™ is a trademark owned byHulu. OTT media asset providers may additionally or alternativelyprovide manifest files described above. In addition to media assetsand/or manifest files, providers of OTT media assets can distributeapplications (e.g., web-based applications or cloud-based applications),or the media asset can be displayed by applications stored onconsumption device 102.

In some embodiments, the audio stream 106 is transmitted into thepresentation of the media asset while the output characteristics areadjusted. For example, the volume for the audio segment where the voiceprofile 106 is identified is modified. In another example, the volumefor the audio segment where the voice profile 106 is identified ismuted. In another example, second audio segment 106 is identifiedconcurrently with first audio segment 104. Consumption device 102 maydetermine or instruct the server to transmit one of the audio segmentsfrom the second audio segment 106 and first audio segment 104. In someembodiments, the server transmits instructions to the user device 102 toprevent transmission by, for example, muting a speaker of user device102 for a predetermined period of time, such as five seconds. After thepredetermined period of time, the system via server 114 may return totransmitting the audio segment.

In some embodiments, the excitement scores are generated based onmachine learning algorithms that rely on real-time video processing oflive gameplay, as well as audio processing - such as converting thecommentators’ audio to text and performing natural language processingto assist a predictive engine in determining an excitement score for anygiven play. Additionally, sentiment analysis can be performed on theaudio portion (e.g., audio associated with sports commentators) toassist in determining the excitement score. For example, thecommentator’s tone and pitch are normal during normal plays as opposedto other plays that might yield a touchdown, field goal, etc. In oneembodiment, different predictive models are utilized based on the sport(e.g., soccer, hockey, football, etc.). Similarly, data can be used tomodel performance of players. Player stats are officially available forvarious leagues and sports and can be relied upon to create such models.The multi-view service supports the management of a plurality of sporttypes. The predictive engine relies on historical data and is initiallypowered by human-labeled scores (training dataset) on a variety of playsand variety of games. The scores are associated with video segments aswell as audio of sports commentators. Predicting the outcome of a playis a classification model and depends on past and current data,including players involved in the play, previous performances,completion percentages, number of yards gained on a play, rushing yards,possession, etc. The training data set is updated after the results ofthe plays are known to keep improving the accuracy of the model. It isimportant to note that different models can be used for different playtypes.

Another method of filtering audio segments may be accomplished bytranscribing a voice signal into corresponding text at the server oraccessing a closed caption data to identify the subject matter. Theserver 114 may transcribe audio segment 104 into corresponding text orclosed captions when not already available for the media asset. Usingnatural language processing, server 114 may determine a language usagelevel. The server may compare the language usage level with preferencesfor user 107 from user preference database 108. User device 102 maydetermine which audio segments of the transcribed text should betransmitted into the media asset and which should be muted based on thecontext of the media asset. For example, if the first media assetincludes a news report, audio spoken by the first media asset may betransmitted, while if the second media asset is a weather report, audiospoken by the second media asset may be not transmitted. Alternativelyor additionally, user device 102 may determine the subject matter ofeach audio segment of the text. Based on preferences to avoid scarystories, crime stories, or traffic stories, as received in a selectionfrom the user at the consumption device 102, user preference database108 may also include subject-matter data as well as actions to performwhen the particular voice profile is identified. For example, the usermay have saved a control action for audio segments that contain somesubject matter or certain people to decrease the volume a specificamount, present closed captions or mute the media asset altogether.

In some embodiments, the “multi-view” service dynamically adapts tobandwidth fluctuation as well. For example, if two games are playingside by side and there’s a sudden drop in bandwidth at the client side,then the multi-view service stops streaming one of the games so that theavailable bandwidth is allocated to just one game. The decision on whichlive stream to keep playing depends on the actions in the game at thattime, as well as user preferences. For example, a red-zone play in GameA takes priority over a mid-field play in Game B if the bandwidthavailable to be reallocated to the client (as a result of ceasing thestreaming of Game B) results in a better picture quality for Game A. Forexample, the red-zone play can then be streamed in UHD if Game B isn’tbeing viewed.

In one embodiment, the multi-view service manages the requests (e.g.,requests for media segments) on behalf of the various players, includinginstantiating additional players, assigning streams to players ofdifferent sizes and locations on the screen, determining which game(s)to keep live streaming, etc. This multi-view service makes suchdeterminations based on information queried from the client as well asremote services, such as a service that provides excitement scores ofgames. The multi-view service can be considered as a proxy between thevideo player(s) and the various streaming sources.

The multi-view service can terminate/pause a stream from a first source(e.g., channel broadcasting Game B) and request media content from asecond channel, e.g., initiate the request on behalf of a video playerfor a second stream (e.g., Game A). Similarly, the multi-view service iscapable of intercepting manifests for various games, and manipulatingthem, e.g., by creating a different manifest that only includesreferences to the segments that should be played, including mergingreferences to content from different manifests into one so that the sameplayer instance can request the content associated with the variousgames intended for any particular user. For example, a first portion ofthe manifest contains references to segments associated with Game Awhile the second portion contains references associated with Game B.

In yet another embodiment, the multi-view service aggregates referencesto segments associated with various games that the user is watching,e.g., segments that depict interesting plays that were not sent to thevideo player due to false negatives of the predictive engine, and makesthem available to the user in a highlight reel, game summary, etc.Additionally, this missed content can be sent to the client when othergames are idle (e.g., no interesting plays are taking place in any ofthe games that the user subscribed to watch through the multi-viewservice).

In one embodiment, the media content associated with games that were notrequested from the video player(s) is buffered and available for theuser. For example, many live games on platforms such as MLB areavailable on demand after the game is over. Such platforms allow usersto rewind through the content throughout the game. That same content canalso be used to create the game summary, highlight reels, etc. Adedicated service provides the excitement scores associated with variousgames, or games of interest to the user, to the multi-view service. Forexample, the multi-view service can subscribe to receive specificupdates for specific games, based on the games that the user selectedfor viewing, or based on user preferences learned over time. Forexample, the multi-view service can subscribe to notificationsconcerning red zone plays, potential score changes, injuries, etc., forspecific teams/games. The term “subscribing” in this context is inaddition to picture quality for Game A. For example, the red-zone playcan then be streamed in UHD if Game B isn’t being viewed.

In some embodiments, the multi-view service manages the requests (e.g.,requests for media segments) regarding various players, includinginstantiating additional players, assigning streams to players ofdifferent sizes and locations on the screen, determining which game(s)to keep live streaming, etc. This “multi-view” service makes suchdeterminations based on information queried from the client as well asremote services, such as a service that provides excitement scores ofgames. The multi-view service can be considered as a proxy between thevideo player(s) (e.g., devices) and the various streaming sources. Themulti-view service can terminate/pause a stream from a first source(e.g., channel broadcasting Game B) and request media content from asecond channel, e.g., initiate the request on behalf of a video playerfor a second stream (e.g., Game A). Similarly, the multi-view service iscapable of intercepting manifests for various games and manipulatingthem, e.g., by creating a different manifest that only includesreferences to the segments that should be played, including mergingreferences to content from different manifests into one so that the sameplayer instance can request the content associated with the variousgames intended for any particular user. For example, a first portion ofthe manifest contains references to segments associated with Game Awhile the second portion contains references associated with Game B.

In yet another embodiment, the multi-view service aggregates referencesto segments associated with various games that the user is watching,e.g., segments that depict interesting plays that were not sent to thevideo player due to false negatives of the predictive engine and makesthem available to the user in a highlight reel, game summary, etc.Additionally, this missed content can be sent to the client when othergames are idle (e.g., no interesting plays are taking place in any ofthe games that the user subscribed to watch through the multi-viewservice). In one embodiment, the media content associated with gamesthat were not requested from the video player(s) are buffered andavailable for the user. For example, many live games on platforms suchas MLB are available on demand after the game is over. Such platformsallow users to rewind through the content throughout the game. That samecontent can also be used to create the game summary, highlight reels,etc.

Dedicated service provides the excitement scores associated with variousgames, or games of interest to the user, to the multi-view service. Forexample, the multi-view service can subscribe to receive specificupdates for specific games, based on the games that the user selectedfor viewing, or based on user preferences learned over time. Forexample, the multi-view service can subscribe to notificationsconcerning red zone plays, potential score changes, injuries, etc., forspecific teams/games. In this context, the term “subscribing” is inaddition to notifications and is used by the multi-view service todetermine which content to show to the user.

FIG. 2A shows a real-time adaptive bitrate transcoding of a media asset,and transmission of transcoded media to a client device over a networkfor playback, which accounts for latency in the network itself, as wellas any additional latency introduced by the transcoder. FIG. 2A is ablock diagram representing the transmission of data broken down to audioplaylist and video playlist and a combined playlist for a first channelbetween a server device (e.g., database 201) and a client device (e.g.,user device 203) in accordance with some embodiments of the disclosure.Database 201, which may be cloud-based server database, may receive arequest for multiple media assets, and in response to the requesttransmit to a client device 203 the multiple media assets. Client device203, which may be a user equipment device (e.g., a set-top box) or amobile device of a user, may transmit a request for a first portion ofthe first media asset and for a first portion of the second media asset.In some embodiments, the database 201 may generate a combined audioplaylist for the two requested media assets. The database 201 may thentransmit the first media asset (e.g., Master Playlist Ch. 1) includingthe audio playlist, video playlist and a combined audio playlist.Similarly, the database 201 may then transmit for the second media asset(e.g., Master Playlist Ch. 2) including the audio playlist, videoplaylist and a combined audio playlist. As shown in FIG. 2B, the serverdevice 201 may transmit a combined audio playlist and video playlist 230(e.g., Master Playlist Ch. 1 and Ch. 2) for the first media asset andthe second media asset in response to the request. In some embodiments,as part of the transmission of the first portion, database 201 may alsotransmit a manifest of audio playlists and a manifest of videoplaylists, each being associated with a bitrate variant, where eachmanifest varies the amount of output data per time segment. Controlcircuitry of the client device 201 may process the transcode latencyvalues to determine a second bitrate at which to request a secondportion and/or a time to transmit the second request or switchtransmitting.

As used herein, a “portion” may be a “segment” or “chunk” of the mediaasset, or any other part of the media asset that represents, contains,or comprises less data than that of the media asset as a whole. A“segment” may comprise a number of seconds of media content for outputon the client device, and may be the minimum unit of media that can beplayed back by the client device 102. For example, a segment maycomprise five seconds of audio and video data representing a five-secondportion of the media asset. A “chunk” may comprise a quantity of dataless than that of a complete segment at a requested bitrate. A segmentmay be divided into a plurality of chunks for transmission using HTTPchunked transfer encoding or any other suitable streaming data transferprotocol.

The server device 201 and client device 203 may be connected via acommunications network, as described below. Each of the server device201 and client device 203 may comprise control circuitry for performingany of the steps, actions, and/or processes described herein. The serverdevice 201 may further include transcoding circuitry, which may be partof the control circuitry, or may be a separate module. The transcodingcircuitry may be implemented in hardware, firmware, or software. Theclient device 203 may further include a display and speaker modules forplayback of transcoded media.

Systems and methods are described herein for a media guidanceapplication that detects a user’s engagement with a second media assetand alerts the user to an event in the second media asset beingpresented on another channel, thereby reminding the user to view themedia asset presented on the user device. The media guidance applicationmay determine when the next important event will occur in the secondmedia asset and determine an estimated usage time of the user devicebased on the application with which the user is engaged on the userdevice. The media guidance application may, depending on the estimatedusage time and the time of the next important event in the second mediaasset, pause the presentation of the first media asset and suggestviewing the second media asset on the display of the user device. Insome embodiments, the media guidance application presents the secondmedia asset in multi-view mode, or overlays the presentation of thesecond media asset over the presentation of the first media asset.

As an example, the user may be watching the movie “Star Wars” on theuser device. During the movie, the system may monitor, based on userpreference, a second media asset. The system may process the secondmedia asset based on object analysis 314, frame analysis 316, subtitleanalysis 318, speech analysis 320 and/or natural language processing322. Based on each of the analyses, the system may recommend via analert to switch to the second media asset for a scene or segment of thesecond media asset. The media guidance application may determine that afirst media asset is being presented on the display and estimate anamount of time when the second media asset will come on. For example,the media guidance application may access a user preference and userhistory to determine what the user prefers to consume. The mediaguidance application may determine when the next important event in thesecond movie occurs. For example, the media guidance application mayaccess metadata of the second “Star Wars” movie and determine that thenext important event (e.g., the death of Obi-Wan Kenobi at the hands ofDarth Vader) will occur in five minutes. The media guidance applicationmay compare the important event and the time to consume the media assetagainst the current media asset presented on the media device. Thesystem may recommend for the important event to be presented by way ofan alert during the presentation of the first media asset. In someembodiments, the media guidance application may present an alert on thesecond device that the user may soon miss an important event.

FIG. 3 shows an exemplary user device and an exemplary alert generatedby the media guidance application for display on the second device 304.In some embodiments, the media guidance application may generate fordisplay, at a user device, a first media asset 302. For example, themedia guidance application may generate for display a football game onthe user device. The football game may be generated for display from abroadcast stream received at the first device, an over-the-top streamingapplication on the user device, or a recording stored in a memory of thefirst device. In some embodiments, the system retrieves details ofusers’ viewing patterns, and preferred characters are learned orinitially captured by user interference from other shows or movies theuser is interested in. Based on a predicted time window of when theevent is likely to occur in the second media asset (e.g., near the endof an Olympic race, halftime of a football match), the system identifieswhen to switch channels to consume the event. The system then configuresthe consumption on a user device of “N” event, where the maximumthreshold of N can be derived based on the end-user device’s hardwarecapabilities.

FIG. 3 further shows a system that performs processing of the mediaassets upstream of the user device. The system includes an analyzer 308that analyzed the varying media assets to identify events that the usermay be interested in. In particular, the analyzer 308 receives the mediaasset details 322, metadata pertaining to the media asset 324 and timeframe 326 and performs object analysis 314, frame analysis 316, subtitleanalysis 318, speech analysis 320 and/or natural language processing322. As part of an object analysis 314, the processing circuitrymonitors for objects within a media asset. In some embodiments, objectanalysis 314 performs an analysis of actors in the media asset. As partof a frame analysis 316, the processing circuitry monitors frames withina media asset for brightness, speed and general layout. In someembodiments, frame analysis 316 performs an analysis of locations withinthe media asset. As part of a subtitle analysis 318, the processingcircuitry monitors the closed captions of the media asset for keywordsthat may interest the user. As part of the speech analysis, 320 controlcircuitry analyzes a base frequency of voice signal and determines aplurality of voice characteristics, such as pitch, intonation, accent,loudness, and speech rate. This data may be stored in association with acharacter or a media asset. As part of natural language processing, 322control circuitry performs an analysis of language and tries todetermine the subject matter discussed. In some embodiments, theprocessing circuitry compares, based on the analysis, each segment ofthe second media asset with user preferences. In some embodiments, theprocessing circuitry determines that a segment of the second media assetmatches the user preferences. When the second media asset segmentmatches the user preferences, the system may prompt an alert for theuser. In some embodiments, the alert is displayed on the user’s device.In some embodiments, the alert is presented on a secondary user deviceor a mobile device. In some embodiments, in response to the alert, theuser device may receive a selection of the event. In some embodiments,in response to selecting to view the event, the media guidanceapplication may generate for display on the user device, in apicture-and-picture mode, the segment of the second media asset.

In some embodiments, the media guidance application may, when performingthe analysis, based on metadata, of a playback time of an importantevent in the media asset, access a database 312 of playback timescorresponding to important events in the media asset. The database maybe stored locally or may be stored on a remote server. The playbacktimes may be relative to the start of the media asset or may be actualtimes based on the time at which the playback of the media asset began.For example, a database 312 for “football games” may indicate importantevents at ten minutes, twenty-five minutes, fifty-three minutes, andeighty-four minutes from the start of the game. The important events aredetermined based on the analysis of the media asset and the userpreferences. For example, the user enjoys watching a certain footballteam in a scoring position. The media guidance application maydetermine, based on the current playback time, the next important eventin the media asset in the database. For example, suppose the currentplayback time for a “football game” is seven minutes. In that case, themedia guidance application may determine that the next important eventoccurs is at three minutes and retrieve three minutes as the playbacktime of the next important event. In some embodiments, the determinationis performed on live programming based on the above-discussed analysis.In such cases, the lead time to identify and mark an event as importantmay be a couple of seconds. As another example, the database for abroadcast of the “Jets game,” which began at 7:00 PM, may indicateimportant events at 7:10 PM (e.g., kickoff) as well as determine animportant event in response to the analysis that is performed on theprogramming. If the current time is 7:47 PM, the media guidanceapplication may determine that the next important event occurs at 7:48PM based on the lag in transmitting the live programming. The mediaguidance application may compare the current time with the timesindicated in the database by first converting both times to a UTCinteger value, or compare the current time’s hour and minute values withthose of the event times indicated in the database.

In some embodiments, the system may present an alert on the display ofthe device for the user to view the important event. In someembodiments, in response to presenting the second media asset to theuser device, the system determines that a playback time ended of thesegment of the second media asset (e.g., the important event ended). Insome embodiments, the system may present textual updates of the secondmedia asset during the presentation of the segment. In some embodiments,the segment is auto-triggered into the display of the user device viaPIP or a dual-view display. Based on the playback of the segment of thesecond media asset ending, the system generates for display on the userdevice the first media asset of the plurality of media assets. Forexample, as the scoring drive in the Jets game ends, the system returnsto the first media asset. In some embodiments, the media guidanceapplication, upon determining that the segment of the second media assethas concluded, automatically switches back to generating for display thefirst media asset.

FIG. 4 shows an exemplary user device 400 and several exemplary displayswith the user device. In some embodiments, the media guidanceapplication may generate a multi-view or quad view display with multiplemedia assets as previously discussed above in relation to multi-view andquad view. The user device 400 includes a first screen 402 presenting afirst media asset, a second screen 408 presenting a second media asset,a third screen 406 presenting closed captions corresponding to the firstmedia asset and a fourth screen 404 presenting closed captionscorresponding to the second media asset.

The methods and systems include receiving a request for a first mediaasset and a second media asset for presentation on a user device. Forexample, a user wants to watch the Jets football game and a Patriotsfootball game on the same television. With the use of a media guidanceapplication, the system generates a user interface including a firstscreen configured for the presentation of the first media asset and asecond screen configured for the presentation of the second media asset.For example, a grid mode is presented with each football game reservedfor one of the boxes in the grid. The system processes an audio manifestfor the first media asset and the second media asset. Each audiomanifest includes a plurality of audio segments, each audio segmentbeing associated with a priority rating. For example, the user parsesthe audio segments into smaller, more consumable audio versions based onpriority bits assigned by a service provider. For example, each segmentreceives a priority rating. The system then compares the priority ratingfor each audio segment from the first media asset with the priorityrating for each audio segment from the second media asset. Based on thecomparing, the system generates a combined audio manifest with audiosegments from the first media asset and the second media asset based onthe audio segment having a higher priority rating. Finally, the systemtransmits for presentation on the user device a video stream of thefirst media asset on the first screen, a video of the second media asseton the second screen and the combined audio manifest.

In some embodiments, the processing circuitry identifies closedcaptioning for each audio segment and inserts the closed caption of themedia asset that is being muted or for which volume is reduced. Forexample, as the user consumes two football games, the first media assetis muted and the audio for the second media asset is presented. At thesame time as the first media asset is muted, the closed captions arepresented to permit the user to read about the video on the first mediaasset. In some embodiments, the presentation on the user device of theclosed caption for each audio segment was determined to have a lowerpriority rating. For example, as the priority of the different segmentschanges, the closed captions are turned on and off for the media assets.In some embodiments, the processing circuitry generates the combinedaudio manifest by identifying a priority rating of a first segment ofthe first media asset and identifying a priority rating of a secondsegment of the second media asset. The first segment of the first mediaasset overlaps in presentation with the second segment of the secondmedia asset. The processing circuitry determines between the firstsegment of the first media asset and the second segment of the secondmedia asset a preferred segment based on a higher priority rating. Theprocessing circuitry interleaves the preferred segment with subsequentaudio segments with a higher priority rating to create a combined audiomanifest. In some embodiments, the processing circuitry arranges eachaudio segment of the first media asset and the second media asset tointerleave the media assets in a sequence.

In some embodiments, generating the combined audio manifest is a userenable mode. In some embodiments, the processing circuitry generates thecombined audio manifest by generating the combined audio segment via anaudio priority analyzer, configured to compare each concurrent audiosegment and switch to a higher priority audio segment. In someembodiments, the priority rating for each segment is biased based on auser preference of the user device.

In some embodiments, the processing circuitry in response totransmitting for presentation on the user device, the video stream ofthe first media asset on the first screen, the video of the second mediaasset on the second screen and the combined audio manifest, receiving aselection for an audio segment from the first media asset. Theprocessing circuitry transmits for presentation on the user device, thevideo stream of the first media asset on the first screen, the video ofthe second media asset on the second screen and the audio segment fromthe first media asset. For example, during the presentation of theimportant event, the user chooses to listen to the audio segment of theother media asset on the display screen.

FIG. 5 shows a real-time audio priority analyzer 510 of a collection 502of audio manifests 504, 506, 508 of a plurality of media assets. FIG. 5is a block diagram representing the compiling of the audio manifest fora plurality of media assets through an audio priority analyzer 510 togenerate a single audio manifest to be presented through speakers 512 ofa user device. The priority ratings may be user-entered,system-generated, system-captured, a third party-captured and/orassigned to each media asset or a combination thereof. For example, anonline blog of sections of a show or a movie may provide a high ratingfor action segments or love segments. The priority rating of each mediasegment may vary based on the totality of the circumstances surroundingthe audio segment. The user may prefer violent and vulgar content. As aresult, the priority rating may be biased/ adjusted based on therespective user when such a scene or segment comes on. Based on thepriority rating, the system transmits a video playlist and a combinedaudio playlist for the first channel between a server device (e.g.,database 201) and a client device (e.g., user device 203) in accordancewith some embodiments of the disclosure.

FIG. 6 is a flowchart representing an illustrative process generating anaudio playlist from a simultaneous presentation of a plurality of mediaassets. The flowchart in FIG. 6 represents a process 600 implemented oncontrol circuitry 904 to generate a playlist from a simultaneouspresentation of a plurality of media assets. It should be noted thatprocess 600 or any action thereof could be performed on, or provided by,any of the devices shown in FIGS. 9-10 . For example, process 600 may beexecuted by control circuitry 904 (FIG. 9 ) as instructed by a mediaguidance application implemented on a user device (e.g., user equipmentdevices 1002, 1004, and/or 1006 (FIG. 10 ) discussed in more detailbelow) in order to generate a playlist from a simultaneous presentationof a plurality of media assets.

At 602, control circuitry 904 may be configured to retrieve, by controlcircuitry, a user preference associated with a user profile. Forexample, database 108 may store user preferences that the user has inputinto the user device 102. In some embodiments, the user preferences areidentified based on viewing history. The user preference may include alist of favorite teams, favorite sports, favorite players, favoriteshows, actors, viewing conditions (e.g., very loud, very quiet), array,or other data structure listing.

At 604, control circuitry 904 may be configured to receive a selectionof a first media asset and a second media asset from the plurality ofmedia assets for presentation on a user device. For example, controlcircuitry 904 may receive a selection to view two different media assets(e.g., football games, Olympic events).

At 606, control circuitry 904 may be configured to parse the respectiveaudio streams of the first media asset and the second media asset toidentify one or more preferred audio segments based on the userpreference. For example, the control circuitry may evaluate the mediaassets to identify segments that may be broken up (e.g., by acommercial, or change in subject matter, change in tone). Controlcircuitry 904 may use the data relating to past media assets to parsethe audio stream of the first media asset into the segments. Similarly,control circuitry 904 may use the data relating to past media assets toparse the audio stream of the second media asset into the segments.Based on the parsed segments of the multiple media assets, the controlcircuitry may identify preferred segments based on comparison to theuser’s preferences.

At 608, control circuitry 904 may be configured to generate the playlistof the identified one or more preferred audio segments. For example,control circuitry 904 may combine the identified audio segments from thefirst and second media assets into a single combined audio stream.

At 610, control circuitry 904 may be configured to generate, forpresentation on the user device, the video stream for each of the firstmedia asset and the second media asset and the playlist of theidentified one or more preferred audio segments. For example, controlcircuitry 904 may generate for television the first media asset (Jetsfootball game) and the second media asset (Patriots football game) alongwith the generated combined audio of both games. That is, as the user iswatching both games on the display, in a dual-view mode, the audiobounces back and more forth from the Jets game and the Patriots gameautonomously with no input from the user. This way, the user may switchher/his attention to the football game where audio is coming from and isfreed from manually switching the channels.

The actions and descriptions of FIG. 6 may be used with any otherembodiment of this disclosure. In addition, the actions and descriptionsdescribed in relation to FIG. 6 may be done in suitable alternativeorders or in parallel to further the purposes of this disclosure. Any ofthese actions may also be skipped or omitted from the process.Furthermore, it should be noted that any of the devices or equipmentdiscussed in relation to FIGS. 9-10 could be used to perform one or moreof the actions in FIGS. 6 and 8 .

FIG. 7 is a flowchart representing an illustrative process foridentifying events in a second media asset and generating the secondmedia asset for the presentation of an event in the second media asset.The flowchart in FIG. 7 represents a process 700 implemented on controlcircuitry 904 to identify events in a second media asset and generatethe second media asset to present an event in the second media asset. Itshould be noted that process 700 or any action thereof could beperformed on, or provided by, any of the devices shown in FIGS. 9-10 .For example, process 700 may be executed by control circuitry 904 (FIG.9 ) as instructed by a media guidance application implemented on a userdevice (e.g., user equipment devices 1002, 1004, and/or 1006 (FIG. 10 )discussed in more detail below) in order to identify events in a secondmedia asset and generate the second media asset for the presentation ofan event in the second media asset.

At 702, control circuitry 904 may be configured to retrieve a userpreference associated with a user profile. For example, database 108 maystore user preferences that the user has input into the user device 102.In some embodiments, the user preferences are identified based onviewing history. The user preference may include a list of favoriteteams, favorite sports, favorite players, favorite shows, actors,viewing conditions (e.g., very loud, very quiet), array, or other datastructure listing.

At 704, control circuitry 904 may be configured to identify, based onthe user preference, a plurality of media assets. For example, controlcircuitry 904 may receive a selection to view a first media asset andidentify a second media asset that the user profile has been consuming.

At 706, control circuitry 904 may be configured to generate a firstmedia asset of the plurality of media assets for display on the userdevice. For example, the media guidance application may process themedia asset and generate video and audio signals for output via display412 and speakers 414, respectively, on the user device.

At 708, control circuitry 904 may be configured to access metadata of asecond media asset, to identify a segment of the second media assetcomprising an event, the event being identified based on the userpreference. For example, control circuitry 904 may access metadatastored in a database pertaining to the media asset. In some examples,the media asset may be a series and may be transmitted over the Internetincluding metadata about certain events within the media asset. Themetadata related to the second media asset may be retrieved from amemory of the second device or cloud-based storage from the contentsource. For example, the metadata may include data fields comprising anindex of events and corresponding playback times. As another example,the metadata may include data fields indicating the importance of theevent represented in the current scene or in the next scene. Controlcircuitry 904 may search the metadata, based on the current playbacktime, for the next scene with an indicated importance level above athreshold importance level.

At 710, control circuitry 904 may be configured to determine, based onthe metadata, a playback time of the segment of the second media assetcomprising the event. For example, control circuitry 904 may add theestimated additional amount of time to the length of the second mediaasset. For example, the metadata may include a timestamp of the event ormay include a relative amount of time from the current playback time tothe important event. Control circuitry 904 may calculate the amount oftime between the current playback time and the playback time of theimportant event. Alternatively, control circuitry 904 may convert thecurrent system time to an integer representing the current system timein Universal Time Code (“UTC”) format. Control circuitry 904 may alsoconvert a broadcast time of the next important event to a UTC integervalue. Control circuitry 904 may then subtract the integer representingthe current time from the integer representing the broadcast time of thenext important event to determine the threshold amount of time. Controlcircuitry 904 may store the threshold amount of time in a variable orother data structure in, for example, storage 908.

At 712, control circuitry 904 may be configured to, during the playbacktime of the segment of the second media asset comprising the event,generate for display on the user device the segment of the second mediaasset. For example, control circuitry 904 may instruct the user device302 to display a pop-up message reminding the user to watch “Footballgame” on the user device because an important play/scene is coming soon.For example, control circuitry 904 may transmit a command to the userdevice 302 such as “ALERT [media_name]” where [media_name] is the titleof the media asset being viewed on the user device, such as “Jetsfootball game.” Depending on the configuration of control circuitry 904,the alert displayed on the second device 302 may include an option torecord the media asset. Control circuitry 904 may, alternatively oradditionally, include an option to pause the media asset in the alert.The alert may also include an option to dismiss the alert without takingany action.

The actions and descriptions of FIG. 7 may be used with any otherembodiment of this disclosure. In addition, the actions and descriptionsdescribed in relation to FIG. 7 may be done in suitable alternativeorders or in parallel to further the purposes of this disclosure. Any ofthese actions may also be skipped or omitted from the process.Furthermore, it should be noted that any of the devices or equipmentdiscussed in relation to FIGS. 9-10 could be used to perform one or moreof the actions in FIGS. 6 and 8 .

FIG. 8 is a flowchart representing an illustrative process forgenerating multiple media assets for presentation on a single userdevice. The flowchart in FIG. 8 represents a process 800 implemented oncontrol circuitry 904 to generate multiple media assets for presentationon a single user device with a combined audio segment. It should benoted that process 800 or any action thereof could be performed on, orprovided by, any of the devices shown in FIGS. 9-10 . For example,process 800 may be executed by control circuitry 904 (FIG. 9 ) asinstructed by a media guidance application implemented on a user device(e.g., user equipment devices 1002, 1004, and/or 1006 (FIG. 10 )discussed in more detail below) in order to generate multiple mediaassets for presentation on a single user device with a combined audiosegment.

At 802, control circuitry 904 may receive a request for a first mediaasset and a second media asset for presentation on a user device. Forexample, control circuitry 904 may receive a selection to view twodifferent media assets (e.g., football games, Olympic events). Forexample, control circuitry 904 may receive, via I/O path 902, a firstmedia asset and a second media asset. Control circuitry 904 may processthe media assets and generate video and audio signals for output viadisplay 912 and speakers 914, respectively.

At 804, control circuitry 904 may be configured to generate a userinterface comprising a first screen configured for presentation of thefirst media asset and a second screen configured for the presentation ofthe second media asset. For example, control circuitry 904 may processthe media assets and generate the first video and second video foroutput via display on the user device 400 and process the audio signals412 for output via speakers 414, respectively.

At 806, control circuitry 904 may be configured to process an audiomanifest for the first media asset and the second media asset. Eachaudio manifest comprises a plurality of audio segments, each audiosegment being associated with a priority rating. For example, thecontrol circuitry may evaluate the media assets to identify segmentsthat may be broken up (e.g., by a commercial, change in subject matter,or change in tone). Control circuitry 904 may use the data relating topast media assets to parse the audio stream of the first media assetinto the segments. Similarly, control circuitry 904 may use the datarelating to past media assets to parse the audio stream of the secondmedia asset into the segments. In some embodiments, the audio segmentsare already identified at the client side and include a ratingassociated with each. For example, an action scene with many thrillingand suspenseful moments may be rated highly compared to a scene withcasual walking and dialogue.

At 808, control circuitry 904 may be configured to compare the priorityrating for each audio segment from the first media asset with thepriority rating for each audio segment from the second media asset. Forexample, control circuitry 904 may compare the media segments of the twoselected media assets playing concurrently. The first media asset isadded to the combined audio manifest based on a priority rating for asegment of the first media segment compared to the second media segment.

At 810, control circuitry 904 may be configured to generate, based onthe comparing, a combined audio manifest with audio segments from thefirst media asset and the second media asset based on the audio segmenthaving the higher priority rating. For example, control circuitry 404may add each audio segment from the comparison with a higher priorityrating.

At 812, control circuitry 904 may be configured to transmit forpresentation on the user device a video stream of the first media asseton the first screen, a video of the second media asset on the secondscreen and the combined audio manifest. For example, control circuitry904 may generate for television the first media asset (Jets footballgame) and the second media asset (Patriots football game) along with thegenerated combined audio manifest of both games converted to audio atthe user device. As the user is watching both games on the display, in adual-view mode, the audio bounces back and forth from the Jets game tothe Patriots game autonomously with no input from the user, based on thepriority ratings received from the client metadata. This way, the usermay switch her/his attention to the football game where audio is comingfrom and is freed from manually switching the channels.

The actions and descriptions of FIG. 8 may be used with any otherembodiment of this disclosure. In addition, the actions and descriptionsdescribed in relation to FIG. 8 may be done in suitable alternativeorders or in parallel to further the purposes of this disclosure. Any ofthese actions may also be skipped or omitted from the process.Furthermore, it should be noted that any of the devices or equipmentdiscussed in relation to FIGS. 9-10 could be used to perform one or moreof the actions in FIGS. 6-7 .

Users may access content and the media guidance application (and itsdisplay screens described above and below) from one or more of theiruser equipment devices. FIG. 9 shows a generalized embodiment ofillustrative user equipment device 900. More specific implementations ofuser equipment devices are discussed below in connection with FIG. 10 .User equipment device 900 may receive content and data via input/output(hereinafter “I/O”) path 902. I/O path 902 may provide content (e.g.,broadcast programming, on-demand programming, Internet content, contentavailable over a local area network (LAN) or wide area network (WAN),and/or other content) and data to control circuitry 904, which includesprocessing circuitry 906 and storage 908. Control circuitry 904 may beused to send and receive commands, requests, and other suitable datausing I/O path 902. I/O path 902 may connect control circuitry 904 (andspecifically processing circuitry 906) to one or more communicationspaths (described below). I/O functions may be provided by one or more ofthese communications paths, but are shown as a single path in FIG. 9 toavoid overcomplicating the drawing.

Control circuitry 904 may be based on any suitable processing circuitrysuch as processing circuitry 906. As referred to herein, processingcircuitry should be understood to mean circuitry based on one or moremicroprocessors, microcontrollers, digital signal processors,programmable logic devices, field-programmable gate arrays (FPGAs),application-specific integrated circuits (ASICs), etc., and may includea multi-core processor (e.g., dual-core, quad-core, hexa-core, or anysuitable number of cores) or supercomputer. In some embodiments,processing circuitry may be distributed across multiple separateprocessors or processing units, for example, multiple of the same typeof processing units (e.g., two Intel Core i7 processors) or multipledifferent processors (e.g., an Intel Core i5 processor and an Intel Corei7 processor). In some embodiments, control circuitry 904 executesinstructions for a media guidance application stored in memory (i.e.,storage 908). Specifically, control circuitry 904 may be instructed bythe media guidance application to perform the functions discussed aboveand below. For example, the media guidance application may provideinstructions to control circuitry 904 to generate the media guidancedisplays. In some implementations, any action performed by controlcircuitry 904 may be based on instructions received from the mediaguidance application.

In client/server-based embodiments, control circuitry 904 may includecommunications circuitry suitable for communicating with a guidanceapplication server or other networks or servers. The instructions forcarrying out the above-mentioned functionality may be stored on theguidance application server. Communications circuitry may include acable modem, an integrated services digital network (ISDN) modem, adigital subscriber line (DSL) modem, a telephone modem, Ethernet card,or a wireless modem for communications with other equipment, or anyother suitable communications circuitry. Such communications may involvethe Internet or any other suitable communications networks or paths(which is described in more detail in connection with FIG. 5 ). Inaddition, communications circuitry may include circuitry that enablespeer-to-peer communication of user equipment devices, or communicationof user equipment devices in locations remote from each other (describedin more detail below).

Memory may be an electronic storage device provided as storage 908 thatis part of control circuitry 904. As referred to herein, the phrase“electronic storage device” or “storage device” should be understood tomean any device for storing electronic data, computer software, orfirmware, such as random-access memory, read-only memory, hard drives,optical drives, digital video disc (DVD) recorders, compact disc (CD)recorders, BLU-RAY disc (BD) recorders, BLU-RAY 3D disc recorders,digital video recorders (DVR, sometimes called a personal videorecorder, or PVR), solid state devices, quantum storage devices, gamingconsoles, gaming media, or any other suitable fixed or removable storagedevices, and/or any combination of the same. Storage 908 may be used tostore various types of content described herein as well as mediaguidance data described above. Nonvolatile memory may also be used(e.g., to launch a boot-up routine and other instructions). Cloud-basedstorage, described in relation to FIG. 10 , may be used to supplementstorage 908 or instead of storage 908.

Control circuitry 904 may include video generating circuitry and tuningcircuitry, such as one or more analog tuners, one or more MPEG-2decoders or other digital decoding circuitry, high-definition tuners, orany other suitable tuning or video circuits or combinations of suchcircuits. Encoding circuitry (e.g., for converting over-the-air, analog,or digital signals to MPEG signals for storage) may also be provided.Control circuitry 904 may also include scaler circuitry for upconvertingand downconverting content into the preferred output format of the userequipment 900. Circuitry 904 may also include digital-to-analogconverter circuitry and analog-to-digital converter circuitry forconverting between digital and analog signals. The tuning and encodingcircuitry may be used by the user equipment device to receive and todisplay, to play, or to record content. The tuning and encodingcircuitry may also be used to receive guidance data. The circuitrydescribed herein, including, for example, the tuning, video generating,encoding, decoding, encrypting, decrypting, scaler, and analog/digitalcircuitry, may be implemented using software running on one or moregeneral purpose or specialized processors. Multiple tuners may beprovided to handle simultaneous tuning functions (e.g., watch and recordfunctions, picture-in-picture (PIP) functions, multiple-tuner recording,etc.). If storage 908 is provided as a separate device from userequipment 900, the tuning and encoding circuitry (including multipletuners) may be associated with storage 908.

A user may send instructions to control circuitry 904 using user inputinterface 910. User input interface 910 may be any suitable userinterface, such as a remote control, mouse, trackball, keypad, keyboard,touch screen, touchpad, stylus input, joystick, voice recognitioninterface, or other user input interfaces. Display 912 may be providedas a stand-alone device or integrated with other elements of userequipment device 900. For example, display 912 may be a touchscreen ortouch-sensitive display. In such circumstances, user input interface 910may be integrated with or combined with display 912. Display 912 may beone or more of a monitor, a television, a liquid crystal display (LCD)for a mobile device, amorphous silicon display, low temperature polysilicon display, electronic ink display, electrophoretic display, activematrix display, electro-wetting display, electrofluidic display, cathoderay tube display, light-emitting diode display, electroluminescentdisplay, plasma display panel, high-performance addressing display,thin-film transistor display, organic light-emitting diode display,surface-conduction electron-emitter display (SED), laser television,carbon nanotubes, quantum dot display, interferometric modulatordisplay, or any other suitable equipment for displaying visual images.In some embodiments, display 912 may be HDTV-capable. In someembodiments, display 912 may be a 3D display, and the interactive mediaguidance application and any suitable content may be displayed in 3D. Avideo card or graphics card may generate the output to the display 912.The video card may offer various functions such as accelerated renderingof 3D scenes and 2D graphics, MPEG-2/MPEG-4 decoding, TV output, or theability to connect multiple monitors. The video card may be anyprocessing circuitry described above in relation to control circuitry904. The video card may be integrated with the control circuitry 904.Speakers 914 may be provided as integrated with other elements of userequipment device 900 or may be stand-alone units. The audio component ofvideos and other content displayed on display 912 may be played throughspeakers 914. In some embodiments, the audio component may be in theform of a manifest file and may be combined from one or more videos. Insome embodiments, the audio may be distributed to a receiver (notshown), which processes and outputs the audio via speakers 914.

The guidance application may be implemented using any suitablearchitecture. For example, it may be a stand-alone applicationwholly-implemented on user equipment device 900. In such an approach,instructions of the application are stored locally (e.g., in storage908), and data for use by the application is downloaded on a periodicbasis (e.g., from an out-of-band feed, from an Internet resource, orusing another suitable approach). Control circuitry 904 may retrieveinstructions of the application from storage 908 and process theinstructions to generate any of the displays discussed herein. Based onthe processed instructions, control circuitry 904 may determine whataction to perform when input is received from input interface 910. Forexample, movement of a cursor on a display up/down may be indicated bythe processed instructions when input interface 910 indicates that anup/down button was selected.

In some embodiments, the media guidance application is aclient/server-based application. Data for use by a thick or thin clientimplemented on user equipment device 900 is retrieved on-demand byissuing requests to a server remote to the user equipment device 900. Inone example of a client/server-based guidance application, controlcircuitry 904 runs a web browser that interprets web pages provided by aremote server. For example, the remote server may store the instructionsfor the application in a storage device. The remote server may processthe stored instructions using circuitry (e.g., control circuitry 904)and generate the displays discussed above and below. The client devicemay receive the displays generated by the remote server and may displaythe content of the displays locally on equipment device 900. This way,the processing of the instructions is performed remotely by the serverwhile the resulting displays are provided locally on equipment device900. Equipment device 900 may receive inputs from the user via inputinterface 910 and transmit those inputs to the remote server forprocessing and generating the corresponding displays. For example,equipment device 900 may transmit a communication to the remote serverindicating that an up/down button was selected via input interface 910.The remote server may process instructions in accordance with that inputand generate a display of the application corresponding to the input(e.g., a display that moves a cursor up/down). The generated display isthen transmitted to equipment device 900 for presentation to the user.

In some embodiments, the media guidance application is downloaded andinterpreted or otherwise run by an interpreter or virtual machine (runby control circuitry 904). In some embodiments, the guidance applicationmay be encoded in the ETV Binary Interchange Format (EBIF), received bycontrol circuitry 904 as part of a suitable feed, and interpreted by auser agent running on control circuitry 904. For example, the guidanceapplication may be an EBIF application. In some embodiments, theguidance application may be defined by a series of JAVA-based files thatare received and run by a local virtual machine or other suitablemiddleware executed by control circuitry 904. In some of suchembodiments (e.g., those employing MPEG-2 or other digital mediaencoding schemes), the guidance application may be, for example, encodedand transmitted in an MPEG-2 object carousel with the MPEG audio andvideo packets of a program.

User equipment device 900 of FIG. 9 can be implemented in system 1000 ofFIG. 10 as user television equipment 1002, user computer equipment 1004,wireless user communications device 1006, or any other type of userequipment suitable for accessing content, such as a non-portable gamingmachine. For simplicity, these devices may be referred to hereincollectively as user equipment or user equipment devices, and may besubstantially similar to user equipment devices described above. Userequipment devices, on which a media guidance application may beimplemented, may function as a standalone device or may be part of anetwork of devices. Various network configurations of devices may beimplemented and are discussed in more detail below.

A user equipment device utilizing at least some of the system featuresdescribed above in connection with FIG. 9 may not be classified solelyas user television equipment 1002, user computer equipment 1004, or awireless user communications device 1006. For example, user televisionequipment 1002 may, like some user computer equipment 1004, beInternet-enabled allowing for access to Internet content, while usercomputer equipment 1004 may, like some television equipment 1002,include a tuner allowing for access to television programming. The mediaguidance application may have the same layout on various different typesof user equipment or may be tailored to the display capabilities of theuser equipment. For example, on user computer equipment 1004, theguidance application may be provided as a website accessed by a webbrowser. In another example, the guidance application may be scaled downfor wireless user communications devices 1006.

In system 1000, there is typically more than one of each type of userequipment device, but only one of each is shown in FIG. 10 to avoidovercomplicating the drawing. In addition, each user may utilize morethan one type of user equipment device and also more than one of eachtype of user equipment device.

In some embodiments, a user equipment device (e.g., user televisionequipment 1002, user computer equipment 1004, wireless usercommunications device 1006) may be referred to as a “second screendevice.” For example, a second screen device may supplement contentpresented on a first user equipment device. The content presented on thesecond screen device may be any suitable content that supplements thecontent presented on the first device. In some embodiments, the secondscreen device provides an interface for adjusting the settings anddisplay preferences of the first device. In some embodiments, the secondscreen device is configured for interacting with other second screendevices or for interacting with a social network. The second screendevice can be located in the same room as the first device, a differentroom from the first device but in the same house or building, or in adifferent building from the first device.

The user may also set various settings to maintain consistent mediaguidance application settings across in-home devices and remote devices.Settings include those described herein, as well as channel and programfavorites, programming preferences that the guidance applicationutilizes to make programming recommendations, display preferences, andother desirable guidance settings. For example, if a user sets a channelas a favorite on, for example, the web site www.Tivo.com on theirpersonal computer at their office, the same channel would appear as afavorite on the user’s in-home devices (e.g., user television equipmentand user computer equipment) as well as the user’s mobile devices, ifdesired. Therefore, changes made on one user equipment device can changethe guidance experience on another user equipment device, regardless ofwhether they are the same or a different type of user equipment device.In addition, the changes made may be based on settings input by a user,as well as user activity monitored by the guidance application.

The user equipment devices may be coupled to communications network1014. Namely, user television equipment 1002, user computer equipment1004, and wireless user communications device 1006 are coupled tocommunications network 1014 via communications paths 1008, 1010, and1012, respectively. Communications network 1014 may be one or morenetworks including the Internet, a mobile phone network, mobile voice ordata network (e.g., a 4G or LTE network), cable network, public switchedtelephone network, or other types of communications networks orcombinations of communications networks. Paths 1008, 1010, and 1012 mayseparately or together include one or more communications paths, suchas, a satellite path, a fiber-optic path, a cable path, a path thatsupports Internet communications (e.g., IPTV), free-space connections(e.g., for broadcast or other wireless signals), or any other suitablewired or wireless communications path or combination of such paths. Path1012 is drawn with dotted lines to indicate that in the exemplaryembodiment shown in FIG. 10 it is a wireless path and paths 1008 and1010 are drawn as solid lines to indicate they are wired paths (althoughthese paths may be wireless paths, if desired). Communications with theuser equipment devices may be provided by one or more of thesecommunications paths, but are shown as a single path in FIG. 10 to avoidovercomplicating the drawing.

Although communications paths are not drawn between user equipmentdevices, these devices may communicate directly with each other viacommunication paths, such as those described above in connection withpaths 1008, 1010, and 1012, as well as other short-range point-to-pointcommunication paths, such as USB cables, IEEE 1394 cables, wirelesspaths (e.g., Bluetooth, infrared, IEEE 802-11x, etc.), or othershort-range communication via wired or wireless paths. BLUETOOTH is acertification mark owned by Bluetooth SIG, INC. The user equipmentdevices may also communicate with each other directly through anindirect path via communications network 1014.

System 1000 includes content source 1016 and media guidance data source1018 coupled to communications network 1014 via communication paths 1020and 1022, respectively. Paths 1020 and 1022 may include any of thecommunication paths described above in connection with paths 1008, 1010,and 1012. Communications with the content source 1016 and media guidancedata source 1018 may be exchanged over one or more communications paths,but are shown as a single path in FIG. 10 to avoid overcomplicating thedrawing. In addition, there may be more than one of each of contentsource 1016 and media guidance data source 1018, but only one of each isshown in FIG. 10 to avoid overcomplicating the drawing. (The differenttypes of each of these sources are discussed below.) If desired, contentsource 1016 and media guidance data source 1018 may be integrated as onesource device. Although communications between sources 1016 and 1018with user equipment devices 1002, 1004, and 1006 are shown as throughcommunications network 1014, in some embodiments, sources 1016 and 1018may communicate directly with user equipment devices 1002, 1004, and1006 via communication paths (not shown) such as those described abovein connection with paths 1008, 1010, and 1012.

Content source 1016 may include one or more types of contentdistribution equipment including a television distribution facility,cable system headend, satellite distribution facility, programmingsources (e.g., television broadcasters, such as NBC, ABC, HBO, etc.),intermediate distribution facilities and/or servers, Internet providers,on-demand media servers, and other content providers. NBC is a trademarkowned by the National Broadcasting Company, Inc., ABC is a trademarkowned by the American Broadcasting Company, Inc., and HBO is a trademarkowned by the Home Box Office, Inc. Content source 1016 may be theoriginator of content (e.g., a television broadcaster, a Webcastprovider, etc.) or may not be the originator of content (e.g., anon-demand content provider, an Internet provider of content of broadcastprograms for downloading, etc.). Content source 1016 may include cablesources, satellite providers, on-demand providers, Internet providers,over-the-top content providers, or other providers of content. Contentsource 1016 may also include a remote media server used to storedifferent types of content (including video content selected by a user),in a location remote from any of the user equipment devices. Systems andmethods for remote storage of content, and providing remotely storedcontent to user equipment are discussed in greater detail in connectionwith Ellis et al., U.S. Pat. No. 7,761,892, issued Jul. 20, 2010, whichis hereby incorporated by reference herein in its entirety.

Media guidance data source 1018 may provide media guidance data, such asthe media guidance data described above. Media guidance data may beprovided to the user equipment devices using any suitable approach. Insome embodiments, the guidance application may be a stand-aloneinteractive television program guide that receives program guide datavia a data feed (e.g., a continuous feed or trickle feed). Programschedule data and other guidance data may be provided to the userequipment on a television channel sideband, using an in-band digitalsignal, using an out-of-band digital signal, or by any other suitabledata transmission technique. Program schedule data and other mediaguidance data may be provided to user equipment on multiple analog ordigital television channels.

In some embodiments, guidance data from media guidance data source 1018may be provided to users' equipment using a client-server approach. Forexample, a user equipment device may pull media guidance data from aserver, or a server may push media guidance data to a user equipmentdevice. In some embodiments, a guidance application client residing onthe user’s equipment may initiate sessions with source 1018 to obtainguidance data when needed, e.g., when the guidance data is out of dateor when the user equipment device receives a request from the user toreceive data. Media guidance may be provided to the user equipment withany suitable frequency (e.g., continuously, daily, a user-specifiedperiod of time, a system-specified period of time, in response to arequest from user equipment, etc.). Media guidance data source 1018 mayprovide user equipment devices 1002, 1004, and 1006 the media guidanceapplication itself or software updates for the media guidanceapplication.

In some embodiments, the media guidance data may include viewer data.For example, the viewer data may include current and/or historical useractivity information (e.g., what content the user typically watches,what times of day the user watches content, whether the user interactswith a social network, at what times the user interacts with a socialnetwork to post information, what types of content the user typicallywatches (e.g., pay TV or free TV), mood, brain activity information,etc.). The media guidance data may also include subscription data. Forexample, the subscription data may identify to which sources or servicesa given user subscribes and/or to which sources or services the givenuser has previously subscribed but later terminated access (e.g.,whether the user subscribes to premium channels, whether the user hasadded a premium level of services, whether the user has increasedInternet speed). In some embodiments, the viewer data and/or thesubscription data may identify patterns of a given user for a period ofmore than one year. The media guidance data may include a model (e.g., asurvivor model) used for generating a score that indicates a likelihooda given user will terminate access to a service/source. For example, themedia guidance application may process the viewer data with thesubscription data using the model to generate a value or score thatindicates a likelihood of whether the given user will terminate accessto a particular service or source. In particular, a higher score mayindicate a higher level of confidence that the user will terminateaccess to a particular service or source. Based on the score, the mediaguidance application may generate promotions that entice the user tokeep the particular service or source indicated by the score as one towhich the user will likely terminate access.

Media guidance applications may be, for example, stand-aloneapplications implemented on user equipment devices. For example, themedia guidance application may be implemented as software or a set ofexecutable instructions which may be stored in storage 908, and executedby control circuitry 904 of a user equipment device 900. In someembodiments, media guidance applications may be client-serverapplications where only a client application resides on the userequipment device, and server application resides on a remote server. Forexample, media guidance applications may be implemented partially as aclient application on control circuitry 904 of user equipment device 900and partially on a remote server as a server application (e.g., mediaguidance data source 1018) running on control circuitry of the remoteserver. When executed by control circuitry of the remote server (such asmedia guidance data source 1018), the media guidance application mayinstruct the control circuitry to generate the guidance applicationdisplays and transmit the generated displays to the user equipmentdevices. The server application may instruct the control circuitry ofthe media guidance data source 1018 to transmit data for storage on theuser equipment. The client application may instruct control circuitry ofthe receiving user equipment to generate the guidance applicationdisplays.

Media guidance system 1000 is intended to illustrate a number ofapproaches, or network configurations, by which user equipment devicesand sources of content and guidance data may communicate with each otherfor the purpose of accessing content and providing media guidance. Theembodiments described herein may be applied in any one or a subset ofthese approaches, or in a system employing other approaches fordelivering content and providing media guidance. The following fourapproaches provide specific illustrations of the generalized example ofFIG. 10 .

In one approach, user equipment devices may present on the displaymultiple media assets, with each media asset having an audio and videostream. In response to requests for two media assets, user equipmentdevices can communicate via communications network 1014 with a remoteserver to process the two media assets and generate a combined audiosegment for the two media assets. The combined audio segment isgenerated based on a user preference. The combined media assets includethe video for each media asset and a single audio segment that combinesthe preferred audio segments from each media asset and is transmitted tothe user equipment device via communications network 1014. Each of themultiple individuals in a single home may operate different userequipment devices on the home network.

In a second approach, users may have a desire to consume events withinmultiple media assets. For example, some users may have favorite actors,or scenes that they prefer to see on their device; however, they arealready consuming a media asset. The system performs an analysis on thesecond media asset including performing an object analysis, a frameanalysis, a subtitle analysis, a speech analysis and NLP to identifysegments of a second media asset that the user prefers to consume. Basedon identifying segments from the second media asset, the system accessesmetadata of a second media asset to determine the playtime. The systemdetermines a time when the segments will be presented and during thetime, the system pauses or switch from generating the first media assetfor presentation to generating the identified segment of the secondmedia asset. In some embodiments, the system generates the second mediaasset in a small window overlaying the first media asset. In someembodiments, the system switched to a dual video presentation, with thesecond media asset being inserted into the presentation and the firstmedia asset being muted.

In a third approach, a user equipment device receives a request for afirst media asset and a second media asset for presentation. The userequipment device generates a user interface where the media assets arepresented simultaneously. An audio manifest for each of the media assetsis processed to identify audio segments for each audio manifest andbased on the audio manifest, the system determines a priority rating foreach audio segment. The priority ratings of each audio segment of afirst media asset are compared against each audio segment of a secondmedia asset, to identify higher priority rating audio segments. Based onidentifying a higher priority rating audio segments, the systemgenerates a combined audio manifest of audio segments that overlap withthe higher priority rating audio segments. The system then transmits forpresentation on the user device, a video stream of the first media asseton the first screen, a video of the second media asset on the secondscreen and the combined audio manifest.

In a fourth approach, a user equipment device may operate in a cloudcomputing environment to access cloud services. In a cloud computingenvironment, various types of computing services for content sharing,storage or distribution (e.g., video sharing sites or social networkingsites) are provided by a collection of network-accessible computing andstorage resources, referred to as “the cloud.” For example, the cloudcan include a collection of server computing devices, which may belocated centrally or at distributed locations, that provide cloud-basedservices to various types of users and devices connected via a networksuch as the Internet via communications network 514. These cloudresources may include one or more content sources 516 and one or moremedia guidance data sources 518. In addition or in the alternative, theremote computing sites may include other user equipment devices, such asuser television equipment 502, user computer equipment 504, and wirelessuser communications device 506. For example, the other user equipmentdevices may provide access to a stored copy of a video or a streamedvideo. In such embodiments, user equipment devices may operatepeer-to-peer without communicating with a central server. The cloud mayperform the processing of the media assets such that the audio files areidentified and tagged based on the priority ratings. In someembodiments, the audio files are uniquely prepared based on userpreferences. For example, a user may select two shows to watch and thesystem accesses user preference to generate a combined audio stream ofaudio segments from both media assets. In some aspects, the user wantsto hear all commentary when the offense is playing for one team. Inanother example, the user wants to hear commentary when teams are likelyto score in the red zone. This way, as the user is watching two shows,the audio switches back and forth based on the user preferences. In someembodiments, the switching is based on the server-identified priorityratings.

The cloud provides access to services, such as content storage, contentsharing, or social networking services, among other examples, as well asaccess to any content described above, for user equipment devices.Services can be provided in the cloud through cloud computing serviceproviders or through other providers of online services. For example,cloud-based services can include a content storage service, acontent-sharing site, a social networking site, or other services viawhich user-sourced content is distributed for viewing by others onconnected devices. These cloud-based services may allow a user equipmentdevice to store content to the cloud and to receive content from thecloud rather than storing content locally and accessing locally storedcontent.

A user may use various content capture devices, such as camcorders,digital cameras with video mode, audio recorders, mobile phones, andhandheld computing devices, to record content. The user can uploadcontent to a content storage service on the cloud either directly, forexample, from user computer equipment 504 or wireless usercommunications device 506 having content capture feature. Alternatively,the user can first transfer the content to a user equipment device, suchas user computer equipment 504. The user equipment device storing thecontent uploads the content to the cloud using a data transmissionservice on communications network 514. In some embodiments, the userequipment device itself is a cloud resource, and other user equipmentdevices can access the content directly from the user equipment deviceon which the user stored the content.

Cloud resources may be accessed by a user equipment device using, forexample, a web browser, a media guidance application, a desktopapplication, a mobile application, and/or any combination of accessapplications of the same. The user equipment device may be a cloudclient that relies on cloud computing for application delivery, or theuser equipment device may have some functionality without access tocloud resources. For example, some applications running on the userequipment device may be cloud applications, i.e., applications deliveredas a service over the Internet, while other applications may be storedand run on the user equipment device. In some embodiments, a user devicemay receive content from multiple cloud resources simultaneously. Forexample, a user device can stream audio from one cloud resource whiledownloading content from a second cloud resource. Or a user device candownload content from multiple cloud resources for more efficientdownloading. In some embodiments, user equipment devices can use cloudresources for processing operations, such as the processing operationsperformed by processing circuitry described in relation to FIG. 9 .

FIG. 11 shows an illustrative diagram of a homogeneous audio combinerexemplary signaling mechanism, in accordance with some embodiments. Thehomogeneous audio combiner may be used to merge or combine audiosegments of two or more channels that are carrying media assets that theuser has elected to consume simultaneously. The homogeneous audiocombiner is a type of metadata schema. Any metadata schema may be usedto describe audio attributes and characteristics of the one or more userequipment devices. The metadata schema may be embedded in a media assetmap table (“PMT”) or other mechanisms during the multiplexing processwhere all channel information is multiplexed into a transport stream. Insome embodiments, for a typical digital broadcasting context, themetadata schema implemented is a homogeneous audio combiner. Though thehomogeneous audio combiner is named “ext_x_independent_segments” themetadata schema may be of any alternate name that uniquely maps achannel audio stream for a first media asset and a second media assetfor overlapping durations. A focus of this disclosure is directed tobroadcast transmission of channels; the same may apply to other forms ofcontent delivery that include but are not limited to IP /Hybrid/OTT(“over the top”). In such instances, the homogeneous audio combiner maytake on a suitable format corresponding to the metadata format supportedby such transmission mediums. The metadata schema (e.g., homogeneousaudio combiner) or its equivalents map a primary audio segment frommultiple media assets to one or more combined audio segments. A primaryaudio segment may be a combination of audio segments from multiplechannels to which a subscriber chooses to tune. The homogeneous audiocombiner then transmits the file to the user equipment device under a“v9/combinedAudio.m3u” format. In some embodiments, an auxiliary audiosegment may be inserted into the combined audio segments in response tonot identifying an audio segment that matches a user’s preferences.

As referred herein, the term “in response to” refers to initiated as aresult of. For example, a first action being performed in response to asecond action may include interstitial steps between the first actionand the second action. As referred herein, the term “directly inresponse to” refers to caused by. For example, a first action beingperformed directly in response to a second action may not includeinterstitial steps between the first action and the second action.

The description and drawings described herein represent exampleconfigurations and do not represent all the implementations within thescope of the claims. For example, the operations and steps may berearranged, combined or otherwise modified. Also, structures and devicesmay be represented in the form of block diagrams to represent therelationship between components and avoid obscuring the describedconcepts. Similar components or features may have the same name but mayhave different reference numbers corresponding to different figures.

Some modifications to the disclosure may be readily apparent to thoseskilled in the art, and the principles defined herein may be applied toother variations without departing from the scope of the disclosure.Thus, the disclosure is not limited to the examples and designsdescribed herein, but is to be accorded the broadest scope consistentwith the principles and novel features disclosed herein.

Computer-readable media includes both non-transitory computer storagemedia and communication media including any medium that facilitates thetransfer of code or data. A non-transitory storage medium may be anyavailable medium that can be accessed by a computer. For example,non-transitory computer-readable media can comprise random access memory(RAM), read-only memory (ROM), electrically erasable programmableread-only memory (EEPROM), compact disk (CD) or other optical diskstorage, magnetic disk storage, or any other non-transitory medium forcarrying or storing data or code.

Also, connecting components may be properly termed computer-readablemedia. For example, if code or data is transmitted from a website,server, or other remote source using a coaxial cable, fiber optic cable,twisted pair, digital subscriber line (DSL), or wireless technology suchas infrared, radio, or microwave signals, then the coaxial cable, fiberoptic cable, twisted pair, DSL, or wireless technology are included inthe definition of medium. Combinations of media are also included withinthe scope of computer-readable media.

The processes described above are intended to be illustrative and notlimiting. One skilled in the art would appreciate that the steps of theprocesses discussed herein may be omitted, modified, combined, and/orrearranged, and any additional steps may be performed without departingfrom the scope of the invention. More generally, the above disclosure ismeant to be exemplary and not limiting. Only the claims that follow aremeant to set bounds as to what the present invention includes.Furthermore, it should be noted that the features and limitationsdescribed in any one example may be applied to any other example herein,and flowcharts or examples relating to one example may be combined withany other example in a suitable manner, done in different orders, ordone in parallel. In addition, the systems and methods described hereinmay be performed in real-time. It should also be noted that the systemsand/or methods described above may be applied to, or used in accordancewith, other systems and/or methods.

1-80. (canceled)
 81. A method comprising: receiving a request for afirst media asset and a second media asset for presentation on a userdevice; in response to receiving the request, generating a userinterface comprising a first screen configured for presentation of thefirst media asset and a second screen configured for presentation of thesecond media asset; processing an audio manifest for the first mediaasset and the second media asset, wherein each audio manifest comprisesa plurality of audio segments, each audio segment being associated witha priority rating; comparing the priority rating for each audio segmentfrom the first media asset with the priority rating for each audiosegment from the second media asset; generating, based on the comparing,a combined audio manifest with audio segments from the first media assetand the second media asset based on the audio segment having a higherpriority rating; and transmitting for presentation, on the user device,a video stream of the first media asset on the first screen, a video ofthe second media asset on the second screen and the combined audiomanifest.
 82. The method of claim 81, further comprising: identifyingclosed caption for each audio segment; and transmitting, forpresentation on the user device, a closed caption for each audio segmentdetermined to have a lower priority rating.
 83. The method of claim 81,wherein the user interface further comprises a third screen configuredto display closed captions for the first media asset; and the methodfurther comprising generating a closed caption for each segment fordisplay on the user device in the third screen.
 84. The method of claim81, wherein generating the combined audio manifest with audio segmentsfrom the first media asset and the second media asset based on the audiosegment having a higher priority rating comprises: identifying apriority rating of a first segment of the first media asset; identifyinga priority rating of a second segment of the second media asset, whereinthe first segment of the first media asset overlaps in presentation withthe second segment of the second media asset; determining between thefirst segment of the first media asset and the second segment of thesecond media asset a preferred segment based on a higher priorityrating; and interleaving the preferred segment with subsequent audiosegments with a higher priority rating to create the combined audiomanifest.
 85. The method of claim 84, wherein the interleaving thepreferred segment with the higher priority rating to create the combinedaudio manifest, comprises arranging each audio segment of the firstmedia asset and the second media asset in a sequence.
 86. The method ofclaim 81, wherein generating the combined audio manifest in a userenable mode.
 87. The method of claim 81, wherein generating the combinedaudio manifest, comprises: generating the combined audio manifest via anaudio priority analyzer, configured to compare each concurrent audiosegment and switch to a higher priority audio segment.
 88. The method ofclaim 81, further comprising receiving an audio manifest comprisingaudio from the first media asset and audio from the second media asset;and in response to determining the audio segment from the second mediaasset has a lower priority rating than the audio segment from the firstmedia asset, automatically adjusting an output characteristic of theaudio segment from the second media asset while the audio segment fromthe first media asset, the video from the first media asset, and thevideo from the second media asset are presented on the user device. 89.The method of claim 81, wherein the priority rating for each segment isbiased based on a user preference of the user device.
 90. The method ofclaim 81, further comprising: in response to transmitting forpresentation on the user device, the video stream of the first mediaasset on the first screen, the video of the second media asset on thesecond screen and the combined audio manifest, receiving a selection foran audio segment from the first media asset; and in response receiving aselection for an audio segment from the first media asset, transmittingfor presentation on the user device, the video stream of the first mediaasset on the first screen, the video of the second media asset on thesecond screen and the audio segment from the first media asset.
 91. Asystem comprising: input/output circuitry configured to: receiving arequest for a first media asset and a second media asset forpresentation on a user device; a control circuitry configured to: inresponse to receiving the request, generating a user interfacecomprising a first screen configured for presentation of the first mediaasset and a second screen configured for presentation of the secondmedia asset; processing an audio manifest for the first media asset andthe second media asset, wherein each audio manifest comprises aplurality of audio segments, each audio segment being associated with apriority rating; comparing the priority rating for each audio segmentfrom the first media asset with the priority rating for each audiosegment from the second media asset; and generating, based on thecomparing, a combined audio manifest with audio segments from the firstmedia asset and the second media asset based on the audio segment havinga higher priority rating; and wherein transmitting for presentation viathe input/output circuitry on the user device, a video stream of thefirst media asset on the first screen, a video of the second media asseton the second screen and the combined audio manifest.
 92. The system ofclaim 91, wherein the control circuitry is further configured to:identify a closed caption for each audio segment; and transmit, forpresentation on the user device, the closed caption for each audiosegment determined to have a lower priority rating.
 93. The system ofclaim 91, wherein the user interface further comprises a third screenconfigured to display closed captions for the first media asset; andwherein the control circuitry is further configured to generate a closedcaption for each segment for display on the user device in the thirdscreen.
 94. The system of claim 91, wherein the control circuitry isfurther configured to generate the combined audio manifest with audiosegments from the first media asset and the second media asset based onthe audio segment having a higher priority rating by: identifying apriority rating of a first segment of the first media asset; identifyinga priority rating of a second segment of the second media asset, whereinthe first segment of the first media asset overlaps in presentation withthe second segment of the second media asset; determining between thefirst segment of the first media asset and the second segment of thesecond media asset a preferred segment based on a higher priorityrating; and interleaving the preferred segment with subsequent audiosegments with a higher priority rating to create the combined audiomanifest.
 95. The system of claim 94, wherein the interleaving thepreferred segment with the higher priority rating to create the combinedaudio manifest, comprises arranging each audio segment of the firstmedia asset and the second media asset in a sequence.
 96. The system ofclaim 91, wherein the control circuitry is further configured togenerate the combined audio manifest in a user enable mode.
 97. Thesystem of claim 91, wherein the control circuitry is further configuredto generate the combined audio manifest, by: generating the combinedaudio manifest via an audio priority analyzer, configured to compareeach concurrent audio segment and switch to a higher priority audiosegment.
 98. The system of claim 91, wherein the control circuitry isfurther configured to: receive an audio manifest comprising audio fromthe first media asset and audio from the second media asset; and inresponse to determining the audio segment from the second media assethas a lower priority rating than the audio segment from the first mediaasset, automatically adjust an output characteristic of the audiosegment from the second media asset while the audio segment from thefirst media asset, the video from the first media asset, and the videofrom the second media asset are presented on the user device.
 99. Thesystem of claim 91, wherein the priority rating for each segment isbiased based on a user preference of the user device.
 100. The system ofclaim 91, wherein the control circuitry is further configured to: inresponse to transmitting for presentation on the user device, the videostream of the first media asset on the first screen, the video of thesecond media asset on the second screen and the combined audio manifest,receive a selection for an audio segment from the first media asset; andin response receiving the selection for an audio segment from the firstmedia asset, transmitting for presentation on the user device, the videostream of the first media asset on the first screen, the video of thesecond media asset on the second screen and the audio segment from thefirst media asset. 101-120. (canceled)